Skip to content

feat(sdk,cli): built-in fact-checker agent role (#789) - #795

Merged
tamirdresher merged 2 commits into
devfrom
squad/789-fact-checker-agent
Apr 4, 2026
Merged

tamirdresher merged 2 commits into
devfrom
squad/789-fact-checker-agent

Conversation

@tamirdresher

@tamirdresher tamirdresher commented Apr 3, 2026 •

Copy link
Copy Markdown
Collaborator

What

Adds fact-checker as a built-in agent role in the engineering role catalog, with a charter template and routing patterns for verification and hallucination detection.

Why

No agent currently validates output quality before delivery. A Haiku model fabricated 38+ family details in one user's research. Teams need a verification gate. Closes #789

How

  • New fact-checker role in catalog-engineering.ts (emoji: 🔍, category: quality)
  • Charter template at templates/fact-checker-charter.md with verification methodology and confidence ratings
  • Added to AGENT_TEMPLATES for init scaffolding
  • Template manifest entry for init/upgrade distribution
  • Routing patterns: fact-check, verify, validate, audit, hallucination, devil's advocate
  • Updated roles tests to be extensible (no hardcoded counts)

Testing

  • npm run build passes
  • npm test passes (8 new tests + 3 updated existing tests)

Docs

  • Changeset entry (fact-checker-agent.md)
  • Built-in roles doc update (will be added in follow-up commit)

Exports

N/A (role catalog is internal, accessed via getRoleById)

Breaking Changes

None — additive only. Existing role tests updated to use >= instead of exact counts.

Waivers

None

@tamirdresher
tamirdresher requested review from Copilot and diberry April 3, 2026 18:50
@github-actions

github-actions Bot commented Apr 3, 2026 •

Copy link
Copy Markdown
Contributor

🛫 PR Readiness Check

ℹ️ This comment updates on each push. Last checked: commit 1392ff2

⚠️ 3 item(s) to address before review

Status Check Details
❌ Single commit 2 commits — consider squashing before review
✅ Not in draft Ready for review
✅ Branch up to date Up to date with dev
❌ Copilot review No Copilot review yet — it may still be processing
✅ Changeset present Changeset file found
✅ Scope clean No .squad/ or docs/proposals/ files
✅ No merge conflicts No merge conflicts
✅ Copilot threads resolved No Copilot review threads
❌ CI passing 15 check(s) still running

This check runs automatically on every push. Fix any ❌ items and push again.
See CONTRIBUTING.md and PR Requirements for details.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@github-actions

github-actions Bot commented Apr 4, 2026 •

Copy link
Copy Markdown
Contributor

🏗️ Architectural Review

⚠️ Architectural review: 1 warning(s), 1 info.

Severity Category Finding Files
🟡 warning bootstrap-area 1 file(s) in the bootstrap area (packages/squad-cli/src/cli/core/) were modified. These files must maintain zero external dependencies. Review carefully. packages/squad-cli/src/cli/core/templates.ts
ℹ️ info template-sync Template files changed in packages/squad-cli/templates/ but not in other template locations. If these templates should stay in sync, consider updating the others too. Changed: packages/squad-cli/templates/, Unchanged: templates/, .squad-templates/, .github/workflows/

Automated architectural review — informational only.

@tamirdresher
tamirdresher requested a review from bradygaster April 4, 2026 21:39
Copilot and others added 2 commits April 5, 2026 00:40
- New fact-checker role in engineering catalog (🔍, quality category)
- Charter template with verification methodology and confidence ratings
- Added to AGENT_TEMPLATES for init scaffolding
- Template manifest entry for init/upgrade distribution
- Routing patterns: fact-check, verify, validate, audit, hallucination
- 8 new tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- BASE_ROLES count: exact 20 → at least 20
- listRoles() count: exact 20 → at least 20
- Attribution test: accept any truthy attribution (not just agency-agents)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@tamirdresher
tamirdresher force-pushed the squad/789-fact-checker-agent branch from 0caf110 to 1392ff2 Compare April 4, 2026 21:40
@github-actions

github-actions Bot commented Apr 4, 2026

Copy link
Copy Markdown
Contributor

🟡 Impact Analysis — PR #795

Risk tier: 🟡 MEDIUM

📊 Summary

Metric Count
Files changed 7
Files added 3
Files modified 4
Files deleted 0
Modules touched 4

🎯 Risk Factors

  • 7 files changed (6-20 → MEDIUM)
  • 4 modules touched (2-4 → MEDIUM)

📦 Modules Affected

root (1 file)
  • .changeset/fact-checker-agent.md
squad-cli (2 files)
  • packages/squad-cli/src/cli/core/templates.ts
  • packages/squad-cli/templates/fact-checker-charter.md
squad-sdk (2 files)
  • packages/squad-sdk/src/config/init.ts
  • packages/squad-sdk/src/roles/catalog-engineering.ts
tests (2 files)
  • test/fact-checker-role.test.ts
  • test/roles.test.ts

This report is generated automatically for every PR. See #733 for details.

@tamirdresher
tamirdresher merged commit 9a53769 into dev Apr 4, 2026
17 checks passed
@bradygaster
bradygaster deleted the squad/789-fact-checker-agent branch August 8, 2026 08:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Built-in Devil's Advocate / Fact Checker agent

2 participants